Nicolás E. Díaz Ferreyra

selected publications

1. MSR ’24

   What Can Self-Admitted Technical Debt Tell Us About Security? A Mixed-Methods Study

   Díaz Ferreyra, Nicolás E., Shahin, Mojtaba,  Zahedi, Mansooreh and 2 more authors

   In Proceedings of the 21st International Conference on Mining Software Repositories (MSR ’24) 2024

   Abs arXiv Bib

   Self-Admitted Technical Debt (SATD) encompasses a wide array of sub-optimal design and implementation choices reported in software artefacts (e.g., code comments and commit messages) by developers themselves. Such reports have been central to the study of software maintenance and evolution over the last decades. However, they can also be deemed as dreadful sources of information on potentially exploitable vulnerabilities and security flaws. Objective: This work investigates the security implications of SATD from a technical and developer-centred perspective. On the one hand, it analyses whether security pointers disclosed inside SATD sources can be used to characterise vulnerabilities in Open-Source Software (OSS) projects and repositories. On the other hand, it delves into developers’ perspectives regarding the motivations behind this practice, its prevalence, and its potential negative consequences. Method: We followed a mixed-methods approach consisting of (i) the analysis of a preexisting dataset containing 94,455 SATD instances and (ii) an online survey with 222 OSS practitioners. Results: We gathered 201 SATD instances through the dataset analysis and mapped them to different Common Weakness Enumeration (CWE) identifiers. Overall, 25 different types of CWEs were spotted across commit messages, pull requests, code comments, and issue sections, from which 8 appear among MITRE’s Top-25 most dangerous ones. The survey shows that software practitioners often place security pointers across SATD artefacts to promote a security culture among their peers and help them spot flaky code sections, among other motives. However, they also consider such a practice risky as it may facilitate vulnerability exploits. Implications: Our findings suggest that preserving the contextual integrity of security pointers disseminated across SATD artefacts is critical to safeguard both commercial and OSS solutions against zero-day attacks.

   @inproceedings{diaz2024satd,
     author = {D\'{i}az Ferreyra, Nicol\'{a}s E. and Shahin, Mojtaba and Zahedi, Mansooreh and Quadri, Sodiq and Scandariato, Riccardo},
     year = {2024},
     title = {What Can Self-Admitted Technical Debt Tell Us About Security? A Mixed-Methods Study},
     booktitle = {Proceedings of the 21st International Conference on Mining Software Repositories (MSR '24)},
   }

2. SNAM ’23

   Cybersecurity Discussions in Stack Overflow: A Developer-Centred Analysis of Engagement and Self-Disclosure Behaviour

   Díaz Ferreyra, Nicolás E., Vidoni, Melina,  Heisel, Maritta and 1 more author

   Social Network Analysis and Mining Dec 2023

   Abs arXiv Bib HTML

   Stack Overflow (SO) is a popular platform among developers seeking advice on various software-related topics, including privacy and security. As for many knowledge-sharing websites, the value of SO depends largely on users’ engagement, namely their willingness to answer, comment or post technical questions. Still, many of these questions (including cybersecurity-related ones) remain unanswered, putting the site’s relevance and reputation into jeopardy. Hence, it is important to understand users’ participation in privacy and security discussions to promote engagement and foster the exchange of such expertise. Objective: Based on prior findings on online social networks, this work elaborates on the interplay between users’ engagement and their privacy practices in SO. Particularly, it analyses developers’ self-disclosure behaviour regarding profile visibility and their involvement in discussions related to privacy and security. Method: We followed a mixed-methods approach by (i) analysing SO data from 1239 cybersecurity-tagged questions along with 7048 user profiles, and (ii) conducting an anonymous online survey (N=64). Results: About 33% of the questions we retrieved had no answer, whereas more than 50% had no accepted answer. We observed that proactive users tend to disclose significantly less information in their profiles than reactive and unengaged ones. However, no correlations were found between these engagement categories and privacy-related constructs such as perceived control or general privacy concerns. Implications: These findings contribute to (i) a better understanding of developers’ engagement towards privacy and security topics, and (ii) to shape strategies promoting the exchange of cybersecurity expertise in SO.

   @article{diaz2023stackoverflow,
     title = {Cybersecurity Discussions in {Stack} {Overflow}: A Developer-Centred Analysis of Engagement and Self-Disclosure Behaviour},
     volume = {14},
     issn = {1869-5469},
     doi = {10.1007/s13278-023-01171-z},
     number = {1},
     journal = {Social Network Analysis and Mining},
     author = {Díaz Ferreyra, Nicolás E. and Vidoni, Melina and Heisel, Maritta and Scandariato, Riccardo},
     month = dec,
     year = {2023},
     pages = {16},
   }

3. JSS ’23

   Simple Stupid Insecure Practices and GitHub’s Code Search: A Looming Threat?

   Russel Go, Ken, Soundarapandian, Sruthi,  Mitra, Aparupa and 2 more authors

   Journal of Systems and Software Dec 2023

   Abs Bib HTML

   Insecure coding practices are a known, long-standing problem in open-source development, which takes on a new dimension with the current capabilities for mining open-source software repositories through version control systems. Although most insecure practices require a sequence of interlinked behaviour, prior work also determined that simpler, one-liner coding practices can introduce vulnerabilities in the code. Such simple stupid insecure practices (SSIPs) can have severe security implications for package-based software systems, as they are easily spread over version-control systems. Moreover, GitHub is piloting regular-expression-based code searches across public repositories through its “Code Search Technology”, potentially simplifying unearthing SSIPs. As an exploratory case study, we focused on popular PyPi packages and analysed their source code using regular expressions (as done by GitHub’s incoming search engine). The goal was to explore how detectable these simple vulnerabilities are and how exploitable “Code Search” technology is. Results show that packages on lower versions are more vulnerable, that “code injection” is the most scattered issue, and that about 20% of the scouted packages have at least one vulnerability. Most concerningly, malicious use of this engine was straightforward, raising severe concerns about the implications of a publically available “Code Search”.

   @article{russel2023jss,
     title = {Simple Stupid Insecure Practices and GitHub’s Code Search: A Looming Threat?},
     journal = {Journal of Systems and Software},
     pages = {111698},
     year = {2023},
     issn = {0164-1212},
     doi = {doi.org/10.1016/j.jss.2023.111698},
     author = {Russel Go, Ken and Soundarapandian, Sruthi and Mitra, Aparupa and Vidoni, Melina and Díaz Ferreyra, Nicolás E.},
   }

4. MSR ’23

   LLMSecEval: A Dataset of Natural Language Prompts for Security Evaluations

   Tony, Catherine, Mutas, Markus,  Díaz Ferreyra, Nicolás E. and 1 more author

   In Proceedings of the 20th International Conference on Mining Software Repositories (MSR ’23) Dec 2023

   Abs arXiv Bib

   Large Language Models (LLMs) like Codex are powerful tools for performing code completion and code generation tasks as they are trained on billions of lines of code from publicly available sources. Moreover, these models are capable of generating code snippets from Natural Language (NL) descriptions by learning languages and programming practices from public GitHub repositories. Although LLMs promise an effortless NL-driven deployment of software applications, the security of the code they generate has not been extensively investigated nor documented. In this work, we present LLMSecEval, a dataset containing 150 NL prompts that can be leveraged for assessing the security performance of such models. Such prompts are NL descriptions of code snippets prone to various security vulnerabilities listed in MITRE’s Top 25 Common Weakness Enumeration (CWE) ranking. Each prompt in our dataset comes with a secure implementation example to facilitate comparative evaluations against code produced by LLMs. As a practical application, we show how LLMSecEval can be used for evaluating the security of snippets automatically generated from NL descriptions.

   @inproceedings{tony2023msrtool,
     author = {Tony, Catherine and Mutas, Markus and D\'{i}az Ferreyra, Nicol\'{a}s E. and Scandariato, Riccardo},
     year = {2023},
     title = {LLMSecEval: A Dataset of Natural Language Prompts for Security Evaluations},
     booktitle = {Proceedings of the 20th International Conference on Mining Software Repositories (MSR '23)},
     doi = {10.1109/MSR59073.2023.00084},
   }

5. CHI ’23

   Regret, Delete, (Do Not) Repeat: An Analysis of Self-Cleaning Practices on Twitter After the Outbreak of the COVID-19 Pandemic

   Díaz Ferreyra, Nicolás E., Shahi, Gautam Kishore,  Tony, Catherine and 2 more authors

   In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems (CHI EA ’23) Dec 2023

   Abs arXiv Bib

   During the outbreak of the COVID-19 pandemic, many people shared their symptoms across Online Social Networks (OSNs) like Twitter, hoping for others’ advice or moral support. Prior studies have shown that those who disclose health-related information across OSNs often tend to regret it and delete their publications afterwards. Hence, deleted posts containing sensitive data can be seen as manifestations of online regrets. In this work, we present an analysis of deleted content on Twitter during the outbreak of the COVID-19 pandemic. For this, we collected more than 3.67 million tweets describing COVID-19 symptoms (e.g., fever, cough, and fatigue) posted between January and April 2020. We observed that around 24% of the tweets containing personal pronouns were deleted either by their authors or by the platform after one year. As a practical application of the resulting dataset, we explored its suitability for the automatic classification of regrettable content on Twitter.

   @inproceedings{diaz2023chilbw,
     author = {D\'{i}az Ferreyra, Nicol\'{a}s E. and Shahi, Gautam Kishore and Tony, Catherine and Stieglitz, Stefan and Scandariato, Riccardo},
     year = {2023},
     title = {Regret, Delete, (Do Not) Repeat: An Analysis of Self-Cleaning Practices on Twitter After the Outbreak of the COVID-19 Pandemic},
     booktitle = {Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems (CHI EA '23)},
     articleno = {246},
     numpages = {7},
     keywords = {deleted tweets, online regrets, crisis communication, privacy, COVID-19, self-disclosure},
     location = {Hamburg, Germany},
     series = {CHI EA '23},
     isbn = {9781450394222},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     url = {https://doi.org/10.1145/3544549.3585583},
     doi = {10.1145/3544549.3585583},
   }

6. CHASE ’23

   Developers Need Protection, Too: Perspectives and Research Challenges for Privacy in Social Coding Platforms

   Díaz Ferreyra, Nicolás E., Imine, Abdessamad,  Vidoni, Melina and 1 more author

   In 16th International Conference on Cooperative and Human Aspects of Software Engineering (CHASE 2023) Dec 2023

   Abs arXiv Bib

   Social Coding Platforms (SCPs) like GitHub have become central to modern software engineering thanks to their collaborative and version-control features. Like in mainstream Online Social Networks (OSNs) such as Facebook, users of SCPs are subjected to privacy attacks and threats given the high amounts of personal and project-related data available in their profiles and software repositories. However, unlike in OSNs, the privacy concerns and practices of SCP users have not been extensively explored nor documented in the current literature. In this work, we present the preliminary results of an online survey (N=105) addressing developers’ concerns and perceptions about privacy threats steaming from SCPs. Our results suggest that, although users express concern about social and organisational privacy threats, they often feel safe sharing personal and project-related information on these platforms. Moreover, attacks targeting the inference of sensitive attributes are considered more likely than those seeking to re-identify source-code contributors. Based on these findings, we propose a set of recommendations for future investigations addressing privacy and identity management in SCPs.

   @inproceedings{diaz2023chase,
     author = {D\'{i}az Ferreyra, Nicol\'{a}s E. and Imine, Abdessamad and Vidoni, Melina and Scandariato, Riccardo},
     year = {2023},
     title = {Developers Need Protection, Too: Perspectives and Research Challenges for Privacy in Social Coding Platforms},
     booktitle = {16th International Conference on Cooperative and Human Aspects of Software Engineering (CHASE 2023)},
     doi = {10.1109/CHASE58964.2023.00019},
   }

7. QRS ’22

   GitHub Considered Harmful? Analyzing Open-Source Projects for the Automatic Generation of Cryptographic API Call Sequences

   Tony, Catherine, Díaz Ferreyra, Nicolás,  and Scandariato, Riccardo

   In 2022 IEEE 22nd International Conference on Software Quality, Reliability and Security Companion (QRS-C) Dec 2022

   arXiv Bib

   @inproceedings{tony2022github,
     author = {Tony, Catherine and D\'{i}az Ferreyra, Nicol\'{a}s and Scandariato, Riccardo},
     booktitle = {2022 IEEE 22nd International Conference on Software Quality, Reliability and Security Companion (QRS-C)},
     title = {GitHub Considered Harmful? Analyzing Open-Source Projects for the Automatic Generation of Cryptographic API Call Sequences},
     year = {2022},
     volume = {},
     number = {},
     pages = {},
     doi = {https://doi.org/10.1109/QRS57517.2022.00094},
   }

8. EuroUSEC ’22

   ENAGRAM: An App to Evaluate Preventative Nudges for Instagram

   Díaz Ferreyra, Nicolás E., Ostendorf, Sina,  Aïmeur, Esma and 2 more authors

   In 2022 European Symposium on Usable Security (EuroUSEC 2022) Dec 2022

   Abs arXiv Bib

   Online self-disclosure is perhaps one of the last decade’s most studied communication processes, thanks to the introduction of Online Social Networks (OSNs) like Facebook. Self-disclosure research has contributed significantly to the design of preventative nudges seeking to support and guide users when revealing private information in OSNs. Still, assessing the effectiveness of these solutions is often challenging since changing or modifying the choice architecture of OSN platforms is practically unfeasible. In turn, the effectiveness of numerous nudging designs is supported primarily by self-reported data instead of actual behavioral information. OBJECTIVE: This work presents ENAGRAM, an app for evaluating preventative nudges, and reports the first results of an empirical study conducted with it. Such a study aims to showcase how the app (and the data collected with it) can be leveraged to assess the effectiveness of a particular nudging approach. METHOD: We used ENAGRAM as a vehicle to test a risk-based strategy for nudging the self-disclosure decisions of Instagram users. For this, we created two variations of the same nudge (i.e., with and without risk information) and tested it in a between-subjects experimental setting. Study participants (N=22) were recruited via Prolific and asked to use the app regularly for 7 days. An online survey was distributed at the end of the experiment to measure some privacy-related constructs. RESULTS: From the data collected with ENAGRAM, we observed lower (though non-significant) self-disclosure levels when applying risk-based interventions. The constructs measured with the survey were not significant either, except for participants’ External Information Privacy Concerns (EIPC). IMPLICATIONS: Our results suggest that (i) ENAGRAM is a suitable alternative for conducting longitudinal experiments in a privacy-friendly way, and (ii) it provides a flexible framework for the evaluation of a broad spectrum of nudging solutions.

   @inproceedings{diaz2022eusec,
     author = {D\'{i}az Ferreyra, Nicol\'{a}s E. and Ostendorf, Sina and A\"{i}meur, Esma and Heisel, Maritta and Brand, Matthias},
     year = {2022},
     doi = {https://doi.org/10.1145/3549015.3555674},
     title = {ENAGRAM: An App to Evaluate Preventative Nudges for Instagram},
     booktitle = {2022 European Symposium on Usable Security (EuroUSEC 2022)},
   }

9. ARES ’22

   SoK: Security of Microservice Applications: A Practitioners’ Perspective on Challenges and Best Practices

   Billawa, Priyanka, Bambhore Tukaram, Anusha,  Díaz Ferreyra, Nicolás E. and 3 more authors

   In International Conference on Availability, Reliability and Security (ARES) Dec 2022
10. MSR ’22

    Vul4J: A Dataset of Reproducible Java Vulnerabilities Geared Towards the Study of Program Repair Techniques

    Bui, Quang-Cuong, Scandariato, Riccardo,  and Díaz Ferreyra, Nicolás E.

    In International Conference on Mining Software Repositories (MSR) Dec 2022
11. EASE ’22

    Conversational DevBots for Secure Programming: An Empirical Study on SKF Chatbot

    Tony, Catherine, Balasubramanian, Mohana,  Díaz Ferreyra, Nicolás E. and 1 more author

    In Evaluation and Assessment in Software Engineering (EASE) Dec 2022

    Abs Bib HTML

    Conversational agents or chatbots are widely investigated and used across different fields including healthcare, education, and marketing. Still, the development of chatbots for assisting secure coding practices is in its infancy. In this paper, we present the results of an empirical study on SKF chatbot, a software-development bot (DevBot) designed to answer queries about software security. To the best of our knowledge, SKF chatbot is one of the very few of its kind, thus a representative instance of conversational DevBots aiding secure software development. In this study, we collect and analyse empirical evidence on the effectiveness of SKF chatbot, while assessing the needs and expectations of its users (i.e., software developers). Furthermore, we explore the factors that may hinder the elaboration of more sophisticated conversational security DevBots and identify features for improving the efficiency of state-of-the-art solutions. All in all, our findings provide valuable insights pointing towards the design of more context-aware and personalized conversational DevBots for security engineering.

    @inproceedings{tony2022ease,
      title = {{Conversational DevBots for Secure Programming: An Empirical Study on SKF Chatbot}},
      booktitle = {Evaluation and Assessment in Software Engineering (EASE)},
      publisher = {ACM},
      year = {2022},
      doi = {https://doi.org/10.1145/3530019.3535307},
      author = {Tony, Catherine and Balasubramanian, Mohana and D\'{i}az Ferreyra, Nicol\'{a}s E. and Scandariato, Riccardo},
    }